Contextual remote control user interface

ABSTRACT

A mobile device is coupled to a computer system configured to access media content sources and play media content items provided thereby on a media device. The mobile device receives a first control specification that includes a plurality of first information items each associated with one of the media content sources and a first action definition. The first information items are concurrently displayed on a remote control user interface of the mobile device. In accordance with a selection of one of the first information items corresponding to a first content source, the mobile device transmits a command of a first action definition corresponding to the selected first information item to the computer system, and receives a second control specification that includes a plurality of second information items. Each second information item is associated with a respective content item provided by the first content source and a second action definition.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/877,130, filed May 18, 2020, which is a continuation of U.S. patentapplication Ser. No. 16/279,872, filed Feb. 19, 2019, which is acontinuation of U.S. patent application Ser. No. 15/928,933, filed Mar.22, 2018, which is a continuation of U.S. patent application Ser. No.13/728,980, filed Dec. 27, 2012, which claims the benefit of U.S. PatentApplication No. 61/667,844, filed Jul. 3, 2012, each of which isincorporated by reference herein in its entirety.

This application is related to U.S. patent application Ser. No.15/249,264, filed Aug. 26, 2016, which is a continuation of U.S. patentapplication Ser. No. 13/728,970, filed Dec. 27, 2012, which claims thebenefit of U.S. Patent Application No. 61/667,844, filed Jul. 3, 2012,each of which is incorporated by reference herein in its entirety.

This application is also related to U.S. patent application Ser. No.16/357,059, filed Mar. 18, 2019, which is a continuation of U.S. patentapplication Ser. No. 15/927,894, filed Mar. 21, 2018, which is acontinuation of U.S. patent application Ser. No. 13/728,980, filed Dec.27, 2012, which claims the benefit of U.S. Patent Application No.61/667,844, filed Jul. 3, 2012, each of which is incorporated byreference herein in its entirety.

TECHNICAL FIELD

The present invention relates generally to providing an adaptable remotecontrol for use in media systems.

BACKGROUND

Modern media display systems often incorporate many different componentsfor playing different types of media content. Moreover, users of mediasystems (such as multi-component home media systems) can now accessmedia content from many different sources, such as broadcast, cable, orsatellite television services, Internet based movie services, Internetbased music services, local media storage, shared media storage, and thelike. Media from multiple different sources is increasingly accessiblethrough individual devices, such as a DVD or BLU-RAY® player that alsoprovides access to Internet based video services. Different displaycomponents and content sources, however, require unique remote controlenvironments and interfaces to enable a user to effectively manipulatethe device or media source. For example, the ideal remote controlinterface for browsing and selecting locally stored photographs to beviewed on a display device is different than the ideal remote controlinterface for browsing and selecting movies from an Internet based movieservice. Traditional remote controls are unable to dynamically adapt tothe particular media source or device that a user wishes to access.Accordingly, it is highly desirable to provide systems and methods forpresenting various remote control interfaces to a user in a system thatprovides access to multiple media devices and sources.

SUMMARY

The above deficiencies and other problems associated with traditionalremote controls are reduced or eliminated by the disclosed methods anddevices.

A computer-implemented method for allowing interaction between devicesis performed at a first device including a processor and memory. Themethod includes transmitting, to a second device, an information itemfor display on the second device and an action definition associatedwith the information item. The action definition includes a command tobe sent by the second device to the first device in response to a userselection of the information item at the second device.

A computer-implemented method for allowing interaction between devicesis performed at a first device including a processor and memory. Themethod includes executing an application that operates interactivelywith a remote control application executing on a second device. Thesecond device includes a touch-screen display. The method includestransmitting, to a second device, an information item for display on thesecond device and an action definition associated with the informationitem. The action definition includes a command to be sent by the seconddevice to the first device in response to a user selection of theinformation item at the second device.

In some implementations, the method further comprises receiving acommand from the second device in response to a selection of theinformation item by the user; and performing a predefined action inresponse to receiving the command.

In some implementations, the predefined action is causing display ofmedia content associated with the information item on a display deviceassociated with the first device.

In some implementations, the media content is one or more of: a video, atelevision station, an audio track, an audio station, and a picture.

In some implementations, the predefined action is a media-device controlaction.

In some implementations, the media-device control action is one or moreof: changing a channel, changing volume level, changing a devicesetting, initiating a record function, adjusting a playback speed, andchanging an operating state of a device.

In some implementations, the information item is a label.

In some implementations, the label includes a graphic.

In some implementations, the label includes text.

In some implementations, the method further comprises transmitting, tothe second device, a collection of information items, and transmitting,to the second device, a display instruction associated with thecollection of information items.

In some implementations, the display instruction indicates that thecollection of information items is to be displayed in list format, atree format, a gallery format, or a hierarchical folder format.

In some implementations, the method further comprises transmitting, tothe second device, a group of collections; and transmitting, to thesecond device, a display instruction associated with the group ofcollections.

In some implementations, the display instruction indicates that arespective collection of the group of collections is to be displayed ina separate tab of a user interface on the second device.

In some implementations, the display instruction indicates that arespective collection of the group of collections is to be displayed ina separate folder of a user interface of the second device.

In some implementations, the information item and the action definitionare incorporated in an HTML file.

In some implementations, the information item is one of a plurality ofinformation items, the action definition is one of a plurality of actiondefinitions, and the plurality of information items and the plurality ofaction definitions are incorporated in the HTML file.

In some implementations, the HTML file specifies a graphical layout forthe plurality of information items and the plurality of actiondefinitions.

In some implementations, the first device is configured to execute oneor more applications.

In some implementations, the one or more applications are one or more ofa web browser, a media browser, and a media player.

In another aspect, the method includes receiving a command from thesecond device in response to a selection of the information item by theuser, and performing a predefined action in response to receiving thecommand. The predefined action is causing display of media contentassociated with the information item on a display device associated withthe first device. The method further includes transmitting, to thesecond device, a collection of information items, and transmitting, tothe second device, a display instruction associated with the collectionof information items. The method further includes transmitting, to thesecond device, a group of collections, and transmitting, to the seconddevice, a display instruction associated with the group of collections.

A method is performed at a mobile device including a processor, memory,a display, and an input device. The method includes receiving, from a TVprocessor, an information item and an action definition associated withthe information item. The action definition includes a command to besent by the mobile device to the TV processor in response to a userselection of the information item. The method further includesdisplaying the information item on the display. The method furtherincludes receiving a selection of the information item from a user viathe input device. The method further includes transmitting the commandfrom the mobile device to the TV processor.

In some implementations, the method further includes receiving, from theTV processor, a display instruction to display media content associatedwith the information item.

In some implementations, the media content is one or more of: a video, atelevision station, an audio track, an audio station, and a picture.

In some implementations, the action definition is one or more ofchanging a channel, changing volume level, changing a device setting,initiating a record function, adjusting a playback speed, and changingan operating state of a device.

In some implementations, information item is a label.

In some implementations, the label includes a graphic.

In some implementations, the label includes text.

In some implementations, the information item is displayed in apredefined format based at least in part on a property of theinformation item.

In some implementations, the information item is one of a collection ofinformation items, and the collection of information items is displayedin a predefined format based at least in part on a property of theinformation items in the collection.

In some implementations, the predefined format is a list format, a treeformat, a gallery format, or a hierarchical folder format.

In some implementations, the method further includes: receiving, fromthe TV processor, a collection of information items; and receiving, fromthe TV processor, a display instruction associated with the collectionof information items.

In some implementations, the method further includes: receiving, fromthe TV processor, a group of collections; and receiving, from the TVprocessor, a display instruction associated with the group ofcollections.

In some implementations, the display instruction indicates that arespective collection of the group of collections is to be displayed ina separate tab of a user interface on the mobile device.

In some implementations, the display instruction indicates that arespective collection of the group of collections is to be displayed ina separate folder of a user interface of the mobile device.

In some implementations, the TV processor is configured to execute oneor more applications.

In some implementations, the one or more applications are one or more ofa web browser, a media browser, and a media player.

In some implementations, the mobile device is a mobile telephone.

In another aspect, the method includes receiving, from the TV processor,a display instruction associated with the information item. The methodfurther includes receiving, from the TV processor, a collection ofinformation items, and receiving, from the TV processor, a displayinstruction associated with the collection of information items. Themethod further includes receiving, from the TV processor, a group ofcollections, and receiving, from the TV processor, a display instructionassociated with the group of collections.

A device includes one or more processors, memory, and one or moreprograms, wherein the one or more programs are stored in the memory andconfigured to be executed by the one or more processors. The programsinclude instructions for: at a first device, transmitting, to a seconddevice, an information item for display on the second device and anaction definition associated with the information item. The actiondefinition includes a command to be sent by the second device to thefirst device in response to a user selection of the information item atthe second device.

In some implementations, the device further includes instructions forperforming any of the above described methods.

A device includes one or more processors, memory, and one or moreprograms, wherein the one or more programs are stored in the memory andconfigured to be executed by the one or more processors. The programsinclude instructions for: at a first device executing an applicationthat operates interactively with a remote control application executingon a second device. The second device includes a touch-screen display.The programs include instructions for: at a first device transmitting,to a second device, an information item for display on the second deviceand an action definition associated with the information item. Theaction definition includes a command to be sent by the second device tothe first device in response to a user selection of the information itemat the second device.

In some implementations, the device further includes instructions forperforming any of the above described methods.

In another aspect, a device includes one or more processors, memory, andone or more programs, wherein the one or more programs are stored in thememory and configured to be executed by the one or more processors. Theprograms include instructions for: receiving, from a TV processor, aninformation item and an action definition associated with theinformation item. The action definition includes a command to be sent bythe mobile device to the TV processor in response to a user selection ofthe information item. The programs also include instructions fordisplaying the information item on the display, receiving a selection ofthe information item from a user via the input device, and transmittingthe command from the mobile device to the TV processor.

A non-transitory computer-readable storage medium stores one or moreprograms comprising instructions that, when executed by a first device,cause the first device to transmit, to a second device, an informationitem for display on the second device and an action definitionassociated with the information item. The action definition includes acommand to be sent by the second device to the first device in responseto a user selection of the information item at the second device.

In some implementations, the non-transitory computer-readable storagemedium stores one or more programs comprising instructions forperforming any of the above described methods.

A non-transitory computer-readable storage medium stores one or moreprograms comprising instructions that, when executed by a first device,cause the first device to transmit, to execute an application thatoperates interactively with a remote control application executing on asecond device. The second device includes a touch-screen display. Theinstructions also cause the first device to transmit, to a seconddevice, an information item for display on the second device and anaction definition associated with the information item. The actiondefinition includes a command to be sent by the second device to thefirst device in response to a user selection of the information item atthe second device.

In some implementations, the non-transitory computer-readable storagemedium stores one or more programs comprising instructions forperforming any of the above described methods.

In another aspect, a non-transitory computer-readable storage mediumstores one or more programs comprising instructions that, when executedby a mobile device, cause the mobile device to receive, from a TVprocessor, an information item and an action definition associated withthe information item. The action definition includes a command to besent by the mobile device to the TV processor in response to a userselection of the information item. The instructions also cause themobile device to: display the information item on the display, receive aselection of the information item from a user via the input device, andtransmit the command from the mobile device to the TV processor.

In some implementations, the non-transitory computer-readable storagemedium stores one or more programs comprising instructions forperforming any of the above described methods.

A method for allowing interaction between devices is performed at afirst device including a processor and memory. The method includesexecuting an application that operates interactively with a remotecontrol application executing on a second device. The second deviceincludes a touch-screen display. The method further includestransmitting, to the second device, one or more information items fordisplay by the second device as one or more selectable display items.Each respective information item is associated with an actiondefinition, the action definition including a command to be sent by thesecond device to the first device in response to a user selection of therespective information item. The method further includes receiving acommand from the second device in response to a selection of a first oneof the selectable display items by the user. The method further includesperforming a predefined action in response to receiving the command.

A device includes one or more processors, memory, and one or moreprograms, wherein the one or more programs are stored in the memory andconfigured to be executed by the one or more processors. The programsinclude instructions for: executing an application that operatesinteractively with a remote control application executing on a seconddevice, the second device including a touch-screen display;transmitting, to the second device, one or more information items fordisplay by the second device as one or more selectable display items,wherein each respective information item is associated with an actiondefinition, the action definition including a command to be sent by thesecond device to the first device in response to a user selection of therespective information item; receiving a command from the second devicein response to a selection of a first one of the selectable displayitems by the user; and performing a predefined action in response toreceiving the command.

A non-transitory computer-readable storage medium stores one or moreprograms comprising instructions that, when executed by a first device,cause the first device to: execute an application that operatesinteractively with a remote control application executing on a seconddevice, the second device including a touch-screen display; transmit, tothe second device, one or more information items for display by thesecond device as one or more selectable display items, wherein eachrespective information item is associated with an action definition, theaction definition including a command to be sent by the second device tothe first device in response to a user selection of the respectiveinformation item; receive a command from the second device in responseto a selection of a first one of the selectable display items by theuser; and perform a predefined action in response to receiving thecommand.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating the infrastructure of a mediasystem, in accordance with some implementations.

FIGS. 2A-B are block diagrams illustrating data structures for use in amedia system with a contextual remote control, in accordance with someimplementations.

FIG. 3 illustrates an exemplary user interface for use in a media systemwith a contextual remote control, in accordance with someimplementations.

FIG. 4 illustrates an exemplary user interface for use in a media systemwith a contextual remote control, in accordance with someimplementations.

FIG. 5 illustrates an exemplary user interface for use in a media systemwith a contextual remote control, in accordance with someimplementations.

FIG. 6 is a block diagram illustrating a structure of a computer system,in accordance with some implementations.

FIG. 7 is a block diagram illustrating a structure of a mobile device,in accordance with some implementations.

FIG. 8 is a block diagram illustrating exemplary applications, inaccordance with some implementations.

FIG. 9 is a flow chart illustrating a process for allowing interactionbetween devices, in accordance with some implementations.

FIG. 10 is a flow chart illustrating a process for allowing interactionbetween devices, in accordance with some implementations.

Like reference numerals refer to corresponding parts throughout thedrawings.

DETAILED DESCRIPTION

In light of the increasing complexity and diversity of media sourcesthat are accessible through modern media systems, it is highly desirableto provide a remote control system that adapts to the particular mediacontent source a user has selected. As noted above, media devices(including computers) are increasingly able to access many differenttypes of media content, including but not limited to, videos (e.g.,movies, home videos, short-form videos, television shows, etc.), audiotracks (e.g., music, podcasts, etc.), television stations (e.g.,satellite, broadcast, and cable television channels), audio/radiostations, and pictures. Moreover, media devices are increasingly able toaccess these types of media content from multiple different sources,such as broadcast, cable, or satellite television sources,Internet-based streaming movie services, Internet-based music services,and the like.

Media devices are increasingly able to provide multiple functions in asingle device. In some cases, media devices are essentially computersystems with operating systems that host and execute multiple individualapplications. These applications may be configured to access distinctmedia sources, or to display media items of specific types. Butdifferent media sources and media types often require uniqueuser-interfaces so that users can browse media content and control thesystem with ease. When remote control devices are provided for use withsuch multi-function computer systems, it is advantageous that they canadapt to a user-interface that is most convenient for that media sourceor type. For example, it may be more convenient for a user to browsephotos on a remote control when they are categorized by date and shownas thumbnails in a “gallery” type view. On the other hand, when browsinga list of movies, it may be more convenient when the movies are listedin alphabetical order by title, and displayed in a list format.Moreover, different media sources or types often require differentcontrol functions to be available to a user. That is, a video viewingapplication will likely require play and pause functions, but will notrequire channel change functions. Similarly, a photo slideshow viewingapplication may require a “photo transition type” selector, whereas astreaming music application will have no use for such a function.Accordingly, a remote control for use with a system that accessesmultiple media sources and types should adapt so that the mostadvantageous user interface is provided to a user. In some of thedisclosed implementations, remote controls have buttons and/or displaysthat are dynamically reconfigurable, as on a mobile phone or a remotewith programmable display or screen. As described below, in someimplementations, the user-interface is based on the application, thetype of media, or the media source that a user is accessing on acomputer system.

Moreover, media devices that can host multiple applications are oftendesigned so that new and different applications can be installed andexecuted by the media device. As new applications are added to thecombination media device, a remote control for use with these types ofmedia devices should also be able to seamlessly adapt to newapplications without a user needing to re-program the remote control.Implementations described herein provide a remote control system that isadaptable to the different applications that may be executed within amedia system.

Attention is directed to FIG. 1, which is a block diagram illustratingthe functional components of a media system 100 in accordance with someimplementations. The media system 100 includes a display device 110, acomputer system 102, and a mobile device 104. As described in detailherein, the computer system 102 stores and executes a variety ofapplications 124 for accessing a variety of media content items. Themobile device 104 communicates with the computer system 102 to generateand display user interfaces (e.g., remote control interfaces) that aretailored to the particular application that is being executed on thecomputer system 102.

In some implementations, the media system 100 includes additional mediacomponents 122, such as TV tuners (e.g., cable or satellite receivers),DVR/VCR/DVD players, local media storage (e.g., hard drives or othernon-volatile storage devices), audio receivers or amplifiers, and thelike. The display device 110 is any device, component, or combination ofcomponents for displaying media content (including audio-visual content)to a viewer, and includes LCD displays, plasma displays, projectors,televisions, speakers, etc. It should be appreciated that the variouscomponents of the media system 100 can be implemented as separatecomponents or as parts of a single component. For example, the computersystem 102 and the display device 110 could be implemented as onecombined component that includes the functions of both the computersystem 102 and the display device 110. For convenience of illustrationand explanation, the media system 100 is described below as beingimplemented as separate components.

The computer system 102 facilitates the presentation of media on thedisplay device 110. For example, in some implementations, the computersystem 102 is a media device that provides access to various mediasources, such as Internet based media sources, locally stored media,etc. The computer system 102 also supports multiple user interfaces anddisplay functions for various types of media content, such as televisionstations/channels, audio tracks (e.g., music, audio books, spoken word,podcasts, etc.), photos/pictures, videos (e.g., movies, home videos,etc.), and textual media. In some implementations, the computer system102 includes applications 124. The applications 124 allow a user toaccess and display these media content items on the display device 110.The applications 124 may include applications that access remotelystored content as well as locally stored content. For example, a photoviewing application may allow a user to view remotely and/or locallystored photos. In some implementations, the computer system 102 includesan operating system that supports the execution of the applications 124,as described below.

The media system 100 also includes a mobile device 104. The mobiledevice 104 includes a display 106, such as an LCD display, and an inputdevice 108, such as physical buttons or a touch-screen interface. Insome implementations, the mobile device 104 is a multi-function devicethat can act as a remote control for the media system 100. In someimplementations, the mobile device 104 executes one or more applicationsthat enable remote control functionality. In some implementations, themobile device 104 is an electronic device, such as a smartphone, acomputer (including tablet computers, laptop computers, desktopcomputers, handheld computers, etc.), digital audio player, gamingcontroller, watch, programmable remote control, and the like. In someimplementations, the mobile device 104 communicates with the computersystem 102 via a network 120. The mobile device 104 is described ingreater detail herein.

The computer system 102 and the mobile device 104 work in conjunction togenerate and present to a user various user interfaces and controlschemes that are tailored to particular applications 124. In someimplementations, as described below, the computer system 102 sends acontrol specification 111 to the mobile device 104. The controlspecification 111 provides the information necessary for the mobiledevice 104 to act as a remote control for a particular mediaapplication. In some implementations, the control specification 111includes information items 112 (e.g., representing media content itemsand/or control functions), action definitions 114 (e.g., commands to beexecuted upon user selection of an information item), and displayinstructions 115 (e.g., where and how to display the information items112). These components of the control specification 111 are described indetail herein. Exemplary data structures of control specifications 111are discussed below in reference to FIGS. 2A-B.

In some implementations, the mobile device 104 is adapted to presentinformation items 112 to a user in a particular format or schemedepending on the type of media content that the information items 112represent. For example, when a user is browsing photographs, the mobiledevice 104 may be adapted to recognize that the browsed items arephotographs, and display the photographs (i.e., information items) onthe mobile device 104 as a gallery of tiled thumbnails. In anotherexample, the mobile device 104 may be adapted to recognize that thebrowsed items are movies, and display the movie titles in analphabetical list. In some implementations, these display schemes areselected when the mobile device 104 recognizes that the user is browsingmedia content of the type “photographs” or “movies.” In these examples,the display scheme is selected based only on the type of content beingbrowsed, without regard to the application 124 that is being executed.

In some implementations, however, the particular display format orscheme that the mobile device 104 implements depends on the type ofapplication 124 that is running on the computer system 102. For example,the computer system 102 may identify that a photo viewing application iscurrently being executed, and instruct the mobile device 104 toimplement display rules (e.g., display rules 126, discussed below) thatare known to be associated with photo viewing applications. In someimplementations, the applications 124 actively specify the particulardisplay format or scheme that should be implemented on the mobile device104. For example, a photo viewing application being executed by thecomputer system 102 may specify that photographs are to be displayed onthe mobile device 104 as thumbnails (e.g., using display instructions115). Various ways of determining and specifying how information items112 are displayed on the mobile device 104 are discussed in detailbelow.

In some implementations, the mobile device 104 provides media-displaycontrols to a user. Media-display controls allow a user to manipulatethe display of and/or interact with applications 124, or with the mediacontent items accessible through the applications 124. Media-displaycontrols include, for example, playback controls, volume controls,channel controls, media content browsing/selection controls, video-gamecontrols, mode controls, device controls, etc. For example, when thecomputer system 102 is executing an application associated with astreaming movie service, the mobile device 104 may display videoplayback controls. When the computer system 102 is executing a photoviewing application, on the other hand, the mobile device 104 maydisplay “slideshow” controls (e.g., play, pause, photo display duration,photo transition type, etc.). Further, when the computer system 102 isexecuting a game application, the mobile device 104 may display a gamecontroller, so that the user can manipulate the game through the mobiledevice 104. In some implementations, the media display controls arespecified by the computer system 102 and/or the applications 124 thatare active on the computer system 102. Any type of control that isavailable on traditional remote controls may be implemented as amedia-display control on the mobile device 104.

Returning to FIG. 1, in some implementations, the computer system 102sends a control specification 111 to the mobile device 104. As describedabove, the control specification 111 provides the information necessaryfor the mobile device 104 to act as a remote control for a particularmedia application. In some implementations, the control specification111 includes information items 112 and action definitions 114.

An information item 112 is information that describes, indicates, orrepresents a media content item, feature, and/or control (such as amedia-display control) associated with an application 124. Informationitems 112 include, but are not limited to, text, graphics (includingicons, pictures, images, photographs, etc.), sounds, lights, and thelike. In some implementations, information items 112 are titles, filenames, thumbnails, and/or descriptions of media content items. (As notedabove, media content items may include videos, television stations,audio tracks, audio/radio stations, pictures, etc.) Information items112 that represent media-display controls may include, for example,text, icons, images, etc., that indicate functions such as “channel up,”“volume up,” “pause video,” etc. In some implementations, as describedbelow, information items 112 are selectable via an input device 108 onthe mobile device 104. For example, an information item 112 may be abutton, hyperlink, actionable text, or some other component that a usermay click, press, or otherwise select (e.g., by clicking or pressing) onthe mobile device 104. As described below, the mobile device 104 can beadapted to send a particular command 116 to the computer system 102 whenan information item 112 is selected by a user.

Information items 112 are sometimes not selectable, but rather provideinformation to a user. For example, an information item 112 may be anicon of a speaker for display near selectable “arrow buttons” for volumecontrol. The speaker icon in this example is not itself selectable, butrather indicates that nearby “arrow buttons” control the volume of adevice, such as the display device 110.

In order to provide a convenient and efficient user experience, anadaptable remote control according to the present implementations shouldbe able to display information items 112 in a format that is clear, andthat is easy to understand, navigate, and control. As noted above, somedisplay formats are better suited to particular types of media contentitems or information items 112. Accordingly, there are several ways forthe media system 100 to determine the appropriate display scheme forgiven information items 112.

In some implementations, the type of media content item that aninformation item 112 represents defines how the information item 112 isto be displayed. Accordingly, in some implementations, display rules 126associate different types of media content items with different displayinstructions. In some implementations, display rules 126 are stored onthe mobile device 104 such that the applications 124 need not senddisplay instructions to the mobile device 104. Rather, the mobile device104 performs a look-up of the display rules 126 to determine where andhow to display the information items 112. In some implementations, thedisplay rules 126 include specifications for any display attribute ofthe information items 112, including type, size, format, color, font,shape, location, etc. For example, display rules 126 may specify thatinformation items 112 representing photographs are to be displayed in athumbnail gallery format, and that information items 112 representingtelevision channels or movie titles are to be displayed in a listformat.

In some implementations, the applications 124 include displayinstructions 115 in the control specification 111. In someimplementations, display instructions 115 indicate where or how, on thedisplay 106 of the mobile device 104, the information items 112 shouldbe displayed (e.g., where on the display 106 a movie title or a control“button” should appear). In some implementations, the displayinstructions 115 include information about the format in whichinformation items 112 should be presented, as discussed above. In someimplementations, the display instructions include specifications for anydisplay attribute of the information items 112, including type, size,format, color, font, shape, location, etc.

In some implementations, both display rules 126 and display instructions115 are used to generate the display scheme or overall layout of themobile device 104 when it is being used as a remote control device forthe computer system 102. In some implementations, the display rules 126are used as default display rules. In some implementations, the defaultdisplay rules are overridden when the control specification 111 includesdisplay instructions 115.

In some implementations, the display scheme or overall layout of aremote control interface is affected by properties of the particularmobile device 104 on which they are to be displayed. For example, thecomputer system 102 may transmit to a mobile device 104 informationitems 112 corresponding to a list of television channels. If the mobiledevice 104 is a smaller device (e.g., a mobile telephone), theinformation items may be displayed in a textual list. If the mobiledevice 104 is a larger device (e.g., a tablet, desktop, or laptopcomputer), the information items 112 may be displayed as thumbnails in atiled arrangement. In some implementations, display instructions 115 aretailored to specific form factors of mobile device 104, such that thedisplay instructions 115 are specific to the size and functionality ofthe mobile device 104. In some implementations, however, displayinstructions 115 are agnostic to the type, size, or functionality of themobile device 104, and the same display instruction 115 is usedregardless of the properties of the particular mobile device 104.

In some implementations, multiple display instructions 115 are includedin the control specification 111 so that different display schemes canbe specified for devices of different size and functionality. Wheremultiple display instructions 115 are provided, either the computersystem 102 (or the applications 124 executed on the computer system 102)or the mobile device 104 may select which display instruction 115 toimplement.

Returning to FIG. 1, the control specification 111 also includes actiondefinitions 114. Action definitions 114 specify a particular action thatshould be performed by the computer system 102 (or an application 124)in response to a user input at the mobile device 104. In someimplementations, an action definition 114 includes or specifies acommand 116 that is to be sent by the mobile device 104 to the computersystem 102 in response to a user selection of an information item 112.Commands 116, discussed in greater detail below, are any information,data, or signal that specifies an action that the computer system 102,or an application 124 executing on the computer system 102, shouldperform. In some implementations, action definitions 114 are associatedwith information items 112 so that a particular action is performed inresponse to a user selection of an information item 112. For example,where an information item 112 is a title of a movie, an associatedaction definition 114 may indicate that selection of that movie titlewill cause that movie to be played. Further, when an information item112 is an icon representing a media-display control (e.g., a “pause”button), an associated action definition 114 may indicate that selectionof the button will cause a video to be paused.

In some implementations, the information items 112, action definitions114, and display instructions 115 are sent to the mobile device 104 asdiscrete data structures and/or specifications. In some implementations,the information items 112, action definitions 114, and displayinstructions 115 are sent to the mobile device 104 as a single, combineddata structure. In some implementations, the control specification 111is incorporated into an HTML document. In some implementations, the HTMLdocument contains information items 112 in the form of textualhyperlinks or selectable graphics, and action definitions 114 in theform of executable functions to be performed in response to a userselection of the textual hyperlinks or selectable graphics in the HTMLdocument. In some implementations, display instructions 115 areincorporated into the HTML document itself in accordance with HTMLstandards.

As described above, each application 124 may provide a completespecification (e.g., using the control specification 111) for theappearance and functionality of the mobile device 104. However, in someimplementations, the mobile device 104 and/or the computer system 102include graphical and/or functional templates for use by theapplications 124. In some implementations, templates are provided wherea portion of the display 106 of the mobile device 104 is configured todisplay predefined playback controls, and another portion of the display106 is freely customizable by the particular application 124.

In some implementations, templates are fully defined user interfaces,including predefined graphical elements, controls, displays, informationitems 112, action items 114, etc. In some implementations, thesetemplates are stored on the mobile device 104, and are activated ordisplayed on the mobile device 104 in response to a command from thecomputer system 102. In some implementations, the computer system 102can request a particular template to be displayed on the mobile device104 based on the application that is being executed on the computersystem 102. For example, if the computer system 102 is executing a videogame, it will send a command to the mobile device 104 requesting displayof a predefined game controller. In other example, if the computersystem 102 is displaying audio/visual media content, it will send acommand to the mobile device 104 requesting display of a predefinedmedia remote control. In some implementations, the commands sent fromthe computer system 102 simply identify a mode of the computer system102, and the mobile device 104 is configured to recognize a particulartemplate for use with that mode. The mode of the computer system 102 maycorrespond to any state or status of the computer system 102, such as atype of application that is being executed (e.g., a video gameapplication, a media player application), or an operating state (e.g.,media playback state, idle state, etc.). In some implementations,information specifying the mode of the computer system 102 is sent tothe mobile device 104 in addition to, instead of, or as part of thecontrol specification 111. Other template configurations, and othermodes and application types, are also contemplated by the presentdisclosure.

FIG. 2A illustrates a data structure 200 of a control specification 111showing the relation of information items 112, action items 114, anddisplay instructions 115 according to some implementations. In thisexample, the data structure 200 is sent to the mobile device 104 tospecify how movie titles are to be displayed, and what actions should betaken in response to a selection of those movie titles. In this example,the movie titles “The Godfather,” “Citizen Kane,” and “The PrincessBride” are textual information items 112 that are to be displayed on themobile device 104. (While the information items 112 are shown as actualtext in FIG. 2A, the information items 112 may be in other formats aswell, including addresses to remotely stored text, graphics, etc.) Thedisplay instructions 115 identify how the information items 112 are tobe displayed on the mobile device 104. In this example, the informationitems 112 are to be displayed in a “list” format. The data structure 200also includes action definitions 114 associated with the movie titles:in response to a selection of “The Godfather,” the command (e.g.,command 116) “Play ‘The Godfather’” is to be sent to the computer system102.

FIG. 2B illustrates another exemplary data structure 210 of a controlspecification 111, where information items 112 correspond tomedia-display controls. In this example, the information items 112 areicons that represent typical playback control buttons (e.g., rewind,stop, play, pause, and fast-forward). Of course, information items 112are not limited to playback controls, and any type of control may beimplemented in similar fashion. The display instructions 115 indicatewhere, on a display 106 of a mobile device 104, the information items112 are to be displayed. While display instructions 115 are representedas {Position “n”}, it is understood that the display instructions 115may include any instructions that define how or where to display theinformation items 112, including coordinate pairs. Reference topredetermined positions (e.g., predetermined positions 306, shown inFIG. 3) in this example illustrate one possible implementation.Returning to data structure 210, action definitions 114 specify thecommands 116 that are to be sent to the computer system 102 in responseto a user selection of one of the playback controls. Commands 116 areany information, data, or signals that specify an action that thecomputer system 102, or an application 124 executing on the computersystem 102, should perform. In data structure 210, the commands 116include {Rewind}, {Stop}, {Play}, {Pause}, and {Fast-Forward}. Thus,when a user selects the information item 112 corresponding to theright-facing single triangle (a typical “play” icon), a command 116{Play} is sent to the computer system 102.

In some implementations, a control specification 111 includes variousdata structures, each defining a different portion or aspect of a remotecontrol user interface. For example, FIG. 3 illustrates an exemplaryuser interface 300 in accordance with some implementations. The userinterface 300 incorporates information from data structures 200 and 210.In a content region 302, the information items 112 from data structure200 are displayed in accordance with the display instructions 115associated therewith. In a control region 304, the information items 112from the data structure 210 are displayed in accordance with the displayinstructions 115 associated therewith. The layout shown in FIG. 3 isprovided as an example layout, and other layouts are also contemplatedby the present disclosure. For example, a user interface 300 need not bedivided into a content region 302 and a control region 304. Userinterfaces 300 may include any number of different regions, or nodelimited regions at all. Moreover, screen positions of informationitems 112 need not correspond to predefined positions (e.g., predefinedpositions 306) on the display 106 of the mobile device 104. Rather,display instructions 115 can specify display locations or other displayproperties by any appropriate means. Accordingly, developers are giventhe flexibility to define an interface that is specifically adapted foruse with their own application 124.

As described above, in some implementations, the computer system 102 (oran application 124 being executed on the computer system 102) sendsmultiple information items 112 to the mobile device 104. In someimplementations, the multiple information items 112 are sent as a groupof multiple information items 112, which may be described as acollection of information items 112. In some implementations,information items 112 each represent a media content item. For example,a collection of information items 112 may correspond to a folder ofdigital photographs, where each information item 112 represents a singlephotograph within that folder. In some implementations, collections ofinformation items 112 are themselves represented by information items112. That is, an information item 112 may refer to an individual mediacontent item (such as a digital photograph), or may represent the foldercontaining multiple media content items (such as a folder containingdigital photographs).

In some implementations, the computer system 102 (or an application 124being executed on the computer system 102) sends multiple collections ofinformation items 112 (or a “collection of collections”) to the mobiledevice 104. In some implementations, the collections are related to oneanother through hierarchical or categorical organization. The mobiledevice 104, in turn, may display the information items 112 representingthe collections (and those representing individual media content items)according to those hierarchical or categorical organizations. Forexample, the computer system 102 may send to the mobile device multiplefolders (e.g., collections), each containing multiple information items112 (e.g., representing individual media content items.) The folders maythen be displayed according to their hierarchical or categoricalorganization. FIGS. 4 and 5, discussed below, illustrate exemplary userinterfaces where collections of information items 112 and collections ofcollections are displayed in accordance with these ideas.

FIG. 4 illustrates an exemplary user interface where collections ofinformation items are displayed on the mobile device 104. In thisexample, the computer system 102 sends information items representingmultiple photo albums (e.g., “2008 Photos,” “2009 Photos,” etc.), wherethe photo albums are organized in a date and/or subject matter hierarchy(e.g., a parent folder “2010 Photos” may contain several sub foldersrepresenting photo albums, such as “vacation,” “kids,” “holidays,”etc.). In some implementations, the display rules 126 or the displayinstructions 115 indicate that parent folders are to be displayed as alist and the sub-folders (corresponding to individual albums) are to bedisplayed as thumbnails. As shown in FIG. 4, parent folders aredisplayed as list 402, and sub-folders are displayed as thumbnails 404.Further, in FIG. 4, the thumbnail 404 that is selected for eachsub-folder may be an image from within the sub-folder. In someimplementations, the selected thumbnail image is the first image withinthe sub-folder according to some ranked order, including alphabeticalorder, chronological order, and the like.

FIG. 5 illustrates an exemplary user interface 500 where a collection ofcollections is displayed on the mobile device 104. In someimplementations, the computer system 102 sends multiple collections ofinformation items 112 (e.g., a collection of collections), where eachcollection includes a different type of information item 112. Forexample, the computer system 102 may send information items 112 groupedinto three collections: one representing TV channels, anotherrepresenting available “on-demand” movies, and another representingpreviously recorded shows. As shown in FIG. 5, the collections are eachdisplayed in a separate tab 502 of the user interface 500, and theindividual information items 112 are displayed in a list 504.

The user interfaces 400 and 500 described with reference to FIGS. 4 and5 are merely exemplary. Other user interfaces are also contemplated byand are consistent with the present disclosure. Indeed, the displayinstructions 115 and display rules 126 disclosed herein are flexibleenough to allow many different user interfaces to be generated by themedia system 100.

FIG. 6 is a block diagram illustrating one implementation of thecomputer system 102 of the media system 100. The computer system 102typically includes one or more processing units (CPUs, sometimes calledprocessors) 602 for executing programs (e.g., programs stored in memory610), one or more network or other communications interfaces 604, memory610, and one or more communication buses 609 for interconnecting thesecomponents. The communication buses 609 may include circuitry (sometimescalled a chipset) that interconnects and controls communications betweensystem components.

The computer system 102 optionally includes a user interface 605comprising a display 606 and input devices 608 (e.g., keyboard, mouse,touch screen, keypad, etc.). In some implementations, the computersystem 102 is coupled to an external display (via the audio/visualinputs and outputs 603) such as the display device 110 of FIG. 1. Insome implementations, the computer system 102 is incorporated into(i.e., in the same housing as) the display 606.

In some implementations, the computer system 102 includes a tuner 607that is capable of receiving over-the-air TV signals. In someimplementations, the tuner 607 receives other types of TV signals, suchas cable, satellite, and/or internet TV signals. In someimplementations, the tuner 607 is configured to receive analog and/ordigital TV broadcasts.

In some implementations, the computer system 102 includes audio/videoinputs and outputs 603. Audio/video inputs and outputs 603 are physicalinterfaces for establishing connections between various components ordevices of a local portion of the media system 100. For example, in someimplementations, the audio/video inputs and outputs 603 includeconnectors for HDMI cables, coaxial cables, VGA cables, optical cables,and the like. In some implementations, the audio/video inputs andoutputs 603 provide a physical interface between the computer system102, the media components 122, and the display device 110.

Memory 610 includes high-speed random access memory, such as DRAM, SRAM,DDR RAM or other random access solid state memory devices; and typicallyincludes non-volatile memory, such as one or more magnetic disk storagedevices, optical disk storage devices, flash memory devices, or othernon-volatile solid state storage devices. Memory 610 optionally includesone or more storage devices remotely located from the CPU(s) 602. Memory610, or alternately the non-volatile memory device(s) within memory 610,comprises a non-transitory computer readable storage medium. In someimplementations, memory 610 or the computer readable storage medium ofmemory 610 stores the following programs, modules, and data structures,or a subset thereof:

-   -   an operating system 612 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   a communication module 614 that is used for connecting the        computer system 102 to other computers (e.g., the mobile device        104) via the one or more communication interfaces 604 (wired or        wireless) and one or more communication networks 120, such as        the Internet, other wide area networks, local area networks,        metropolitan area networks, and so on;    -   a user interface module 616 that receives commands from the user        via the input devices 608 (or external input device, such as the        mobile device 104) and generates user interface objects in the        display device 606, and/or a display device 110 coupled to the        computer system 102 via the audio/video inputs and outputs 603;    -   an application module 618 that includes and is configured to        execute one or more applications 124;    -   a control specification module 620 that generates and stores        control specification(s) 111 for transmission to the mobile        device 104; and    -   a command receipt module 622 that receives commands 116 from the        mobile device 104, and provides the commands 116 to an active        application 124.

In some implementations, the programs or modules identified abovecorrespond to sets of instructions for performing a function describedin this application. The sets of instructions can be executed by one ormore processors (e.g., the CPUs 602). The above identified modules orprograms (i.e., sets of instructions) need not be implemented asseparate software programs, procedures or modules, and thus varioussubsets of these programs or modules may be combined or otherwisere-arranged in various implementations. In some implementations, memory610 stores a subset of the modules and data structures identified above.Furthermore, memory 610 may store additional modules and data structuresnot described above.

Although FIG. 6 shows a “computer system,” FIG. 6 is intended more asfunctional description of the various features which may be present in acomputer system than as a structural schematic of the implementationsdescribed herein. In practice, and as recognized by those of ordinaryskill in the art, items shown separately could be combined and someitems could be separated. For example, some items shown separately inFIG. 6 could be implemented together on a computer system and singleitems could be implemented by one or more modules, or one or morecomputer systems. The actual number of computer systems and modules usedto implement a computer system 102, and how features are allocated amongthem, will vary from one implementation to another.

In some implementations, the application module 618 includes, and isconfigured to execute, one or more applications 124-i, described withreference to FIG. 8. In some implementations, the one or moreapplications 124-i include a video player/browser application 124-1, aTV viewer/browser 124-2, a music player/browser application 124-3, a webbrowser application 124-4, an interactive game application 124-5, and/ora photo viewer/browser application 124-6. In some implementations, theone or more applications 124-i operate in conjunction with the controlspecification module 620 to generate a control specification 111.

In some implementations, the applications 124 are provided by entitiesthat also provide media content access services. For example, a videoviewing application may be created by, supplied by, and/or associatedwith a company that offers subscription-based access to a collection ofvideo content. In some implementations, the computer system 102 executesthe applications 124 from various media access services, such as videoplayers from multiple different subscription services. In someimplementations, the applications 124 are not associated with mediaaccess services, but rather facilitate access to a user's own mediacontent, such as locally stored photographs and/or movies. It isunderstood that the reference to different media “applications” in thepresent application is intended as more of a functional description ofthe features of a particular media access paradigm, and does notnecessarily describe specific programmatic or functional boundaries. Thefunctionality associated with the separately described applications124-i may be included in a single application or system.

In some implementations, the control specification module 620 operatesin conjunction with the application module 618 (or the applications 124)to generate control specifications 111 for respective applications 124.In some implementations, the control specification module 620 includesone or more control specifications 111 including information items 112,action definitions 114, and display instructions 115. In someimplementations, the control specification 111 includes only a subset ofthe items described. For example, in some implementations, the one ormore control specifications 111 include only information items 112 andaction definitions 114, as described above.

In some implementations, the control specification module 620 identifiesan active application 124, and generates and/or identifies a controlspecification 111 for that application. The control specification module620 then transmits the control specification 111 to the mobile device104. As noted above, the control specification 111 may include one ormore of information items 112, action definitions 114, and displayinstructions 115 that the mobile device 104 can use to generate aremote-control interface for whatever application 124 is being executed.

In some implementations, the control specification module 620communicates with the mobile device 104 via the communications module614 and the communication interface(s) 604. In some implementations, theapplications 124 communicate the control specification 111 (and/orinformation items 112, action definitions 114, and display instructions115) directly to the mobile device 104. In some implementations, theapplications 124 share a common control specification module 620. Insome implementations, each application 124 includes its own controlspecification module 620.

In some implementations, the control specification module 620 receivesone or more of the information items 112, action definitions 114, anddisplay instructions 115 from the applications 124 in order to generatethe control specification 111. In some implementations, the controlspecification module 620 identifies one or more aspects of anapplication 124 and automatically generates a control specification 111for that application 124. For example, the control specification module620 may be able to determine that a video player/browser application isexecuting, and identify titles of videos that are accessible to theapplication and that should be displayed to the user on the mobiledevice 104. The control specification module 620 creates informationitems 112 that correspond to the identified videos. The informationitems 112 are sent to the mobile device 104 for display to a user. Insome implementations, the control specification module 620 identifiesinformation that is currently displayed by a respective application onthe display device 110, and creates information items 112 based on thatinformation. For example, a video player/browser application maydisplay, on the display device 110, a list of available movies. Thecontrol specification module 620 can recognize that the displayed list(or a subset thereof) should be sent to the mobile device 104 forviewing and/or selection by a user. The control specification module 620creates information items 112 based on the list of movies, and sends theinformation items 112 (optionally in conjunction with action definitions114 and/or display instructions 115) to the mobile device 104.

In some implementations, the control specification module 620automatically associates action definitions 114 with information items112. For example, the control specification module 620 can identifyinformation items 112, as described above, and determine that certaintypes of information items 112 should be associated with certain actiondefinitions 114. For example, if an application 124 is configured todisplay movie titles, and those movies are associated with movie filesavailable at uniform resource indicators (URIs), the controlspecification module 620 will associate the action of retrieving themovie file from the URI (i.e., the action definition) with the title ofthe move that is displayed to the user (i.e., the information item).

The control specification module 620 may additionally identifymedia-control functions that are associated with an active application.Continuing the video player/browser example, the control specificationmodule 620 may identify that the video player can respond to (orrequires) controls such as “play,” “stop,” “pause,” etc. The controlspecification module 620 may include action definitions 114 for theidentified functions, which will be sent to the mobile device 104 toappear as control buttons. In some implementations, the controlspecification module 620 associates those action definitions 114 withinformation items 112 (such as icons associated with the above describedfunctions) in the control specification 111.

The computer system 102 also includes a command receipt module 622. Thecommand receipt module 622 receives commands 116 from the mobile device104, and provides the commands 116 to an active application 124. Asdescribed above, commands 116 are any information, data, or signal thatspecifies an action that the computer system 102, or an application 124executing on the computer system 102, should perform. Commands 116 arespecified by action definitions 114 that are sent to the mobile device104, and may be received by the command receipt module 622 via thecommunications module 614 and/or the user interface module 616. In someimplementations, the commands 116 are passed to an application 124 thatis being executed by the computer system 102, so that the actionassociated with (or specified by) the command 116 can be performed. Forexample, if the command receipt module 622 receives a command 116associated with the action “Play” while a video player/browser is beingexecuted, the command receipt module 622 will pass the “Play” command tothe video player, which may cause a selected video to be displayed onthe display device 110.

FIG. 7 is a block diagram illustrating one implementation of the mobiledevice 104. The mobile device 104 typically includes one or moreprocessing units (CPUs, sometimes called processors) 702 for executingprograms (e.g., programs stored in memory 710), one or more network orother communications interfaces 704, memory 710, and one or morecommunication buses 709 for interconnecting these components. Thecommunication buses 709 may include circuitry (sometimes called achipset) that interconnects and controls communications between systemcomponents.

The mobile device 104 optionally includes a user interface 705comprising a display 106 and an input device 108 (e.g., keyboard, mouse,touch screen, keypad, joystick, button, track pad, etc.).

Memory 710 includes high-speed random access memory, such as DRAM, SRAM,DDR RAM or other random access solid state memory devices; and typicallyincludes non-volatile memory, such as one or more magnetic disk storagedevices, optical disk storage devices, flash memory devices, or othernon-volatile solid state storage devices. Memory 710 optionally includesone or more storage devices remotely located from the CPU(s) 702. Memory710, or alternately the non-volatile memory device(s) within memory 710,comprises a non-transitory computer readable storage medium. In someimplementations, memory 710 or the computer readable storage medium ofmemory 710 stores the following programs, modules, and data structures,or a subset thereof:

-   -   an operating system 712 that includes procedures for handling        various basic system services and for performing hardware        dependent tasks;    -   a communication module 714 that is used for connecting the        mobile device 104 to other computers (e.g., the computer system        102) via the one or more communication interfaces 704 (wired or        wireless) and one or more communication networks 120, such as        the Internet, other wide area networks, local area networks,        metropolitan area networks, and so on;    -   a user interface module 716 that receives commands from the user        via the input device(s) 108 and generates user interface objects        in the display 106;    -   an application module 718 that includes and is configured to        execute one or more applications 719;    -   a remote control module 720 that receives and stores control        specification(s) 111 received from the computer system 102; and    -   a command module 722 that receives user selections of        information items 112 and sends commands 116 (corresponding to        the commands specified in the action definitions 114) to the        computer system 102.

In some implementations, the programs or modules identified abovecorrespond to sets of instructions for performing a function describedin this application. The sets of instructions can be executed by one ormore processors (e.g., the CPUs 702). The above identified modules orprograms (i.e., sets of instructions) need not be implemented asseparate software programs, procedures or modules, and thus varioussubsets of these programs or modules may be combined or otherwisere-arranged in various implementations. In some implementations, memory710 stores a subset of the modules and data structures identified above.Furthermore, memory 710 may store additional modules and data structuresnot described above.

FIG. 7 describes the mobile device 104, which can be a remote control,smart phone, tablet computer, or any other electronic device that canexecute a program to control media content played on the display device110 of FIG. 1. However, FIG. 7 is intended more as functionaldescription of the various features which may be present in a mobiledevice (or other type of device) than as a structural schematic of theimplementations described herein. In practice, and as recognized bythose of ordinary skill in the art, items shown separately could becombined and some items could be separated. For example, some itemsshown separately in FIG. 7 could be implemented together on a mobiledevice (or other devices, such as computer system 102) and single itemscould be divided up and implemented by one or more modules, or one ormore other devices. The actual number of devices and modules used toimplement a mobile device 104, and how features are allocated amongthem, will vary from one implementation to another.

In some implementations, the application module 718 includes, and isconfigured to execute, one or more applications 719. The applications719 may be any type of application, including but not limited toapplications for entertainment, social networking, productivity, email,navigation, photography, etc. In some implementations, the mobile device104 is a general purpose computing device, such that applications 719may include any software adapted for use on the mobile device 104.

The remote control module 720 includes instructions and data forhandling operations related to using the mobile device 104 as a remotecontrol for the computer system 102. In some implementations, the remotecontrol module 720 receives a control specification 111 from thecomputer system 102 or an application 124 being executed thereon. Insome implementations, the control specification 111 includes informationitems 112, action definitions 114, and/or display instructions 115, asdescribed above. In some implementations, the remote control module 720includes display rules 126 that specify how information items 112 are tobe displayed when display instructions 115 are not included in thecontrol specification 111, as described above.

In some implementations, the remote control module 720 uses the controlspecification 111 to generate a user interface (e.g., user interface300, 400, or 500) for display to the user on the display 106. The remotecontrol module 720 may cooperate with the user interface module 716 togenerate and/or display the user interface. In some implementations, theuser interface is generated in accordance with the control specification111. For example, the information items 112 may be processed andformatted for display on the mobile device 104 in accordance with thedisplay instructions 115 or the display rules 126, as described above inrelation to FIGS. 3-5.

In some implementations, the remote control module 720 includes acommand module 722. The command module 722 receives user-selections ofinformation items 112 via the input devices 108 (in conjunction with theuser interface module 716), and sends commands 116 to the computersystem 102. In some implementations, in response to receiving a userselection of an information item 112, the command module 722 identifiesthe action definition 114 associated with that information item 112. Insome implementations, the command module 722 then identifies a command116 in the action definition 114, and transmits the command 116 to thecomputer system 102. In some implementations, the command module 722stores multiple commands 116, and looks-up an appropriate command 116based on the action definition 114 associated with a user-selectedinformation item 112. In some implementations, the command module 722formats a command 116 into a data structure appropriate for transmissionto the computer system 102.

In some implementations, the command module 722 sends the commands 116to the computer system 102 via the communications interfaces 704 and thecommunications module 714. In some implementations, the commands 116 aresent using any appropriate communication protocol or method, includingbut not limited to infrared, wife, cellular, and/or local peer-to-peerconnections, etc.

Attention is now directed to FIG. 9, which is a flow diagramillustrating a process 900 for allowing interaction between devices,such as the computer system 102 and the mobile device 104, according tosome implementations. Such methods may be useful for allowing a user tocontrol multiple applications with a single remote control device.

The method 900 is performed at a computer system having one or moreprocessors and memory. In some implementations, the method 900 isperformed by the control specification module 620 and the commandreceipt module 622 of the computer system 102. An application isexecuted (901), where the application operates interactively with aremote control application executing on a second device. In someimplementations, the second device includes a touch-screen display. Insome implementations, the second device is a mobile device 104 asdescribed above. Applications 124 are also described above, and mayinclude media player/browser applications, such as video players andphoto viewers.

In some implementations, an information item is transmitted (902) to asecond device for display on the second device. An action definition isalso transmitted (902) to the second device, where the action definitionis associated with the information item. The action definition includesa command to be sent by the second device to the first device inresponse to a user selection of the information item at the seconddevice. Information items 112 and action items 114 are described indetail above.

In some implementations, a command is received (904) from the seconddevice in response to a selection of the information item by the user.As described above, a command 116 may be sent from the mobile device 104to the computer system 102 in response to a user selection of aninformation item 112. In some implementations, a predefined action isperformed (906) in response to receiving the command.

In some implementations, a collection of information items istransmitted (908) to the second device. In some implementations, adisplay instruction associated with the collection of information itemsis transmitted (910) to the second device. For example, a displayinstruction (e.g., display instruction 115 or display rule 126) mayindicate that each information item 112 in the collection of informationitems should be displayed as a thumbnail in a gallery format.

In some implementations, a group of collections is transmitted (912) tothe second device. In some implementations, a display instructionassociated with the group of collections is transmitted (914) to thesecond device. For example, a display instruction (e.g., displayinstruction 115 or display rule 126) may indicate that each collectionis to be displayed within a separate tab, and the information items 112within each collection are to be displayed in a list format.

Attention is now directed to FIG. 10, which is a flow diagramillustrating a process 1000 for allowing interaction between devices,such as the computer system 102 and the mobile device 104, according tosome implementations.

The method 1000 is performed at a computer system having one or moreprocessors, memory, a display, and an input device. In someimplementations, the method 1000 is performed by the remote controlmodule 720 of the mobile device 104. An information item and an actiondefinition associated with the information item are received (1002) froma TV processor. In some implementations, the TV processor corresponds toa computer system 102 as described above. The action definition includesa command to be sent by the mobile device to the TV processor inresponse to a user selection of the information item. The informationitem is displayed (1004) on the display. The information item (e.g.,information item 112) may be displayed in accordance with a displayinstruction 115 or a display rule 126, as described above. A selectionof the information item is received (1006) from a user via the inputdevice. The command is transmitted (1008) from the mobile device to theTV processor.

In some implementations, a display instruction associated with theinformation item is received (1010) from the TV processor. Displayinstructions (e.g., display instruction 115 or display rule 126) arediscussed in detail above.

In some implementations, a collection of information items is received(1012) from the TV processor. In some implementations, a displayinstruction associated with the collection of information items isreceived (1014) from the TV processor. For example, a displayinstruction (e.g., display instruction 115 or display rule 126) mayindicate that each information item 112 in the collection of informationitems should be displayed as a thumbnail in a gallery format.

In some implementations, a group of collections is received (1016) fromthe TV processor. In some implementations, a display instructionassociated with the group of collection is received (1018) from the TVprocessor. For example, a display instruction (e.g., display instruction115 or display rule 126) may indicate that each collection is to bedisplayed within a separate tab, and the information items 112 withineach collection are to be displayed in a list format.

Each of the methods described herein may be governed by instructionsthat are stored in a non-transitory computer readable storage medium andthat are executed by one or more processors of one or more electronicdevices (e.g., computer system 102, mobile device 104). Each of theoperations shown in FIGS. 9-10 may correspond to instructions stored ina computer memory or computer readable storage medium.

The foregoing description, for purpose of explanation, has referencedspecific implementations. However, the illustrative discussions aboveare not intended to be exhaustive or to limit the disclosed ideas to theprecise forms disclosed. Many modifications and variations are possiblein view of the above teachings. The implementations were chosen anddescribed in order to best explain the principles and practicalapplications of the disclosed ideas, to thereby enable others skilled inthe art to best utilize them in various implementations with variousmodifications as are suited to the particular use contemplated.

Moreover, in the preceding description, numerous specific details areset forth to provide a thorough understanding of the presented ideas.However, it will be apparent to one of ordinary skill in the art thatthese ideas may be practiced without these particular details. In otherinstances, methods, procedures, components, and networks that are wellknown to those of ordinary skill in the art are not described in detailto avoid obscuring aspects of the ideas presented herein.

What is claimed is:
 1. A computer-implemented method, comprising: at acomputer system that is coupled to a media device and facilitates themedia device to output media content provided by a plurality of mediacontent sources: executing an application that accesses the plurality ofmedia content sources and outputs media content provided thereby,wherein the application operates interactively with a remote controlapplication executed on a client device that is distinct and remote fromthe computer system; obtaining an information item corresponding to oneor more media content items that are provided by the plurality of mediacontent sources accessible via the computer system, wherein theinformation item is configured to be displayed by the remote controlapplication on the client device; transmitting the information item fromthe computer system to the client device; and in response to a userinput received at the client device and related to the information item,executing a command in relation to the one or more media content itemscorresponding to the plurality of media content sources.
 2. Thecomputer-implemented method of claim 1, wherein the one or more mediacontent items include an audio track, and wherein the plurality of mediacontent sources include an Internet-based audio channel with which theaudio track is associated.
 3. The method of claim 1, further comprising:incorporating by the computer system the information item and thecommand in an HTML file; and transmitting the command associated withthe information item to the client device.
 4. The method of claim 1,further comprising: in response to the user input, receiving by thecomputer system the command from the client device to perform aspecified action with respect to the one or more media content itemscorresponding to the information item.
 5. The method of claim 1, whereinexecuting the command further comprises: in response to the user input,controlling, by the computer system, the media device to play the one ormore media content items.
 6. The method of claim 1, further comprising:retrieving, by the client device, display rules, wherein the displayrules provide instructions for displaying the information item in a userinterface of the client device; and displaying the information item inthe user interface of the client device according to the display rules.7. The method of claim 1, wherein the media device is a speaker device.8. The method of claim 7, wherein executing the command includesaltering playback of the audio track on the media device.
 9. The methodof claim 7, wherein the command associated with the information itemcomprises at least one of: a Play command; a Stop command; and a Pausecommand.
 10. The method of claim 1, wherein the information itemincludes information describing the one or more media content items, andthe information describing the one or more media content items furthercomprises: a title of the media content items; and an image associatedwith the media content items.
 11. The method of claim 10, wherein theinformation item further includes information describing theInternet-based audio channel with which the audio track is associated.12. A computer system, coupled to a media device and facilitates themedia device to output media content provided by a plurality of mediacontent sources, comprising: one or more processors; memory storing oneor more programs, wherein the one or more programs are configured to beexecuted by the one or more processors and include instructions for:executing an application that accesses the plurality of media contentsources and outputs media content provided thereby, wherein theapplication operates interactively with a remote control applicationexecuted on a client device that is distinct and remote from thecomputer system; obtaining an information item corresponding to one ormore media content items that are provided by the plurality of mediacontent sources accessible via the computer system, wherein theinformation item is configured to be displayed by the remote controlapplication on the client device; transmitting the information item fromthe computer system to the client device; and in response to a userinput received at the client device and related to the information item,executing a command in relation to the one or more media content itemscorresponding to the plurality of media content sources.
 13. Thecomputer system of claim 12, wherein the one or more programs furthercomprise instructions for: retrieving, by the client device, displayrules, wherein the display rules provide instructions for displaying theinformation item in a user interface of the client device; anddisplaying the information item in the user interface of the clientdevice according to the display rules.
 14. The computer system of claim12, wherein the one or more programs further comprise instructions for:sending a display instruction associated with the information item,wherein the display instruction indicates that the one or more mediacontent items are to be displayed in a list format, a tree format, agallery format, or a hierarchical folder format.
 15. The computer systemof claim 12, wherein the information item corresponds to a single mediacontent item provided by the plurality of media content sources or afolder containing multiple media content items provided by the pluralityof media content sources.
 16. The computer system of claim 12, whereinthe information item includes information describing the one or moremedia content items, which is configured to be displayed by the remotecontrol application on the client device.
 17. The computer system ofclaim 16, wherein the information describing one or more media contentitems is selected from a group consisting of a title, a file name, athumbnail, and description of the one or more media content items.
 18. Anon-transitory computer-readable storage medium storing one or moreprograms, the one or more programs comprising instructions, which whenexecuted by a processor of a computer system, cause the computer systemto perform operations for: at the computer system that is coupled to amedia device and facilitates the media device to output media contentprovided by a plurality of media content sources: executing anapplication that accesses the plurality of media content sources andoutputs media content provided thereby, wherein the application operatesinteractively with a remote control application executed on a clientdevice that is distinct and remote from the computer system; obtainingan information item corresponding to one or more media content itemsthat are provided by the plurality of media content sources accessiblevia the computer system, wherein the information item is configured tobe displayed by the remote control application on the client device;transmitting the information item from the computer system to the clientdevice; and in response to a user input received at the client deviceand related to the information item, executing the command in relationto the one or more media content items corresponding to the plurality ofmedia content sources.
 19. The non-transitory computer-readable storagemedium of claim 18, the one or more programs further comprisinginstructions for concurrently: displaying the information item in a userinterface of the client device, wherein the user interface includes acontent region and a control region that are distinct from each other;and displaying one or more predefined playback controls in the controlregion, independently of the plurality of media content sources, wheninformation describing the one or more media content items is displayedin the content region.
 20. The non-transitory computer-readable storagemedium of claim 19, wherein the one or more predefined playback controlsare represented by a subset of a plurality of icons for rewinding,stopping, playing, pausing, and fast-forwarding media content.
 21. Thenon-transitory computer-readable storage medium of claim 19, wherein thecontrol region is fixed on the user interface of the client device, andconfigured to display the one or more predefined playback controlsaccording to a template, and the content region of the user interface iscustomized dynamically based on the plurality of media content sources.